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National Conference 
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Vocational and Technical 
Education 



Baker Hotel 
Dallas, Texas 
March 5, 1969 



INTENT AND PURPOSES OF PART I OF THE 
VOCATIONAL EDUCATION AMENDMENTS OF 1968 



Melvin L. Barlow 

Professor of Education 
University of California, Los Angeles 



The major task confronting the Federal Board for Vocational 
Education in 1917 was the preparation of instructional material. 

Curricula for vocational education did not exist; these had to be 
created and there were few models to follow. 

Curriculum objectives in 1917 were reasonably clear. The end 
product was the productive worker- -at the entry level for beginners in 
the labor force, and increased mobility for the employed worker. These 
basic curriculum objectives are true in 1969, but complications arise 
because of the diverse nature of the groups to be served, and because 
of the wide range of occupations included within the purview of vocational 

education. 

The vocational education curriculum has been sensitive to social 
conditions - -World War I, the great depression of the 1930's, World War 
II, and the contemporary emphasis upon people who have not been well 
treated by society. The second area of curriculum sensitivity has been 
the general state of technology. The dynamism of social conditions and 
technology creates the necessity for attention to change in the vocational 
education curriculum. When social and technical change are rapid, as 
is true of the present period, curriculum change in vocational and 
technical education is commanding, urgent, and imperative. 

From 1917 to the Age of Technology 

At first the major effort in development of instructional material 
was undertaken by the Federal Board for Vocational Education. A large 
staff was retained by the Board for several years to produce the material 
representative of the vocational content. Later, when the program grew 
substantially throughout the nation, this procedure was no longer practical. 
Teacher educators in vocational education then focused attention upon 
"occupational analysis" as a means of helping teachers produce their 
own instructional material. 
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Curriculum development, in its larger sense, did not exist. In 
the high school at least half of the day was devoted to general studies 
and half to vocational studies with no more than accidental relationship 

\ 

between the two. 

Teacher prepared instructional materials, with each teacher 
responsible for his own materials, worked fine for many. years. In 
some instances even today it is the only way to approach a particular 
problem. However, as enrollment in vocational education expanded 
total reliance upon teacher prepared materials became a complicated 
matter. Teacher education institutions began to stockpile good examples 
of curriculum materials and new teachers spent their time updating such 
materials and adapting the materials to their programs of instruction. 
Exchange of materials from teacher to teacher 1 , school to school, and 
state to state became common practice. Within a particular school the 
teacher was left to his own devices, except in larger districts where 
assistance was available from curriculum specialists. 

Then came World War II, and everything changed. The dramatic 
development of instructional materials, in order to prepare more than 
eight million people to work in production in defense of the Nation, 
created new ideas and desires related to curriculum development. Special 
task forces, immediately following World War II, prepared instructional 
materials for special instructional areas. A number of states organized 
curriculum laboratories, and national conferences and workshops were 
conducted in recognition of the imperative need in the total area of 
curriculum development, including a vast expansion in preparation of 
curriculum materials. 

The story at this point becomes considerably more familiar as 
we move into the contemporary period. The need for curriculum 
development and instructional materials continued unabated. The Division 
of Vocational and Technical Education, U. S. Office of Education made 
valiant attempts to solve some of the curriculum problems, and the 
publishers throughout the nation became visible in their desire to participate 
in the realm of published materials. From a variety of sources we began 
to see innovative ideas become practice. But the total effort in curriculum 
development and in preparation of instructional materials was but a small 
drop in the bucket. 
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The Ape of Vocational Education 



In 1961, President Kennedy appointed a Panel of Consultants on 
Vocational Education to study the total area of vocational education and 
to report findings and recommendations. The Panel, in its report 
Education For a Changing World of Work, discussed in some detail the 
problem, plight, and need for curriculum development and for preparation 
of instructional materials. The Panel's recommendations were quite 

clear. 



"it is recommended that the production of in structional 
materials for vocational courses be recognized as v ital to an 

effective nati onal program and that -- 

1. One or more instructional material 

laboratories be established to produce and 
distribute vocational instructional materials. 

a. Programed learning aids, visual aids, 
and newer methods of the presentation 
and use of materials should be considered 
in the production of instructional 
materials. 

b. All materials developed should be made 
available to private publishers for 
maximum distribution. 

2. It be a responsibility of the U. S. Office 
of Education through the Division of Vocational 
and Technical Education to-- 

a. Establish and administer instructional 
materials laboratories through contractual 
arrangements with a State department of 
education, a college, a university, ora 
large school district. 

b. Develop policies for the operation, 
coordination between centers, production 
of materials, and distribution of the 
materials produced in these centers. 

c. Finance the operation of these centers. 

3. An adequate quantity and an appropriate 
quality of instructional supplies, tools, instruments, 
and equipment be recognized as essential to good 
instruction. Standards of evaluation should 
consider the quantity and quality of supplies, „ « 
tools, instruments, and equipment available. 
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The intent of the Panel concerning instructional material 
development did get into the Vocational Education Act of 1963, but it 
was lost in a listing of other imperatives. The Act did not feature the 
need for curriculum and instructional materials development to an extent 
commensurate with the Panel's expectations. Despite later exemplary 
efforts of the Division of Vocational and Technical Education, U. S. 

Office of Education, and similar efforts of the states, the actual progress 
in curriculum development fell far short of expectations and needs. 



Five years later, in 1967, the matter of curriculum development 
came to the attention of the Advisory Council on Vocational Education. 
The Council supported the Panel's earlier curriculum views and in the 
general report of the Council, Vocational Education: The Bridge Between 
Man and His Work, the following recommendation was made: 



"IT IS RECOMMENDED, That there be established 
two to four centers for curriculum development in 
vocational education . 

At present, some 12 curriculum centers are 
operated by the States, usually in cooperation with 
universities. Each of these centers has developed 
curriculum materials for the occupations most 
commonly taught in vocational education. Very 
little time or money has been spend on each of these, 
the result being that we have many poor sets of materials 



for teaching the more common occupations. For the 
less frequently taught occupations, little or no 
curriculum materials are available. There is need 
for two or three well-developed sets of curriculum 
materials for each of the occupational fields. This 
would give each school a choice, and it would still 
prevent waste and unnecessary duplication. 

Probably 10 times as much money has been 
spent on curriculum materials for physics (taken by 
5 percent of the high school students) as has been 
spent on the 100 or more occupations commonly 
taught in vocational education. M 2 



The Council's report provided guidance to the Congress in 
preparation of the Amendments of 1968. This time, however, the Act of 
1968 made curriculum development in vocational and technical education 
a special issue. The details of Part I of the Act delineates clearly the 
intent of the Congress, and expresses in general terms the intent of 
the Advisory Council on Vocational Education. 





\ 
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2 . 

3. 

4. 

5. 



Paraphrasing Part I of the Act of 1968, it says in effect: 

Curriculum development is important. ... . 

The curriculum development task for vocational education is 

complicated in a variety of ways. 

The purpose of Part I is to provide the ways and means of 
achieving the desired program of curriculum development. 

$10 000, 000 is authorized for Fiscal Year 1970. 

The Commissioner of Education has a number of choices 
to make in order to implement this section. 

He can: . ... 

a. Make grants to colleges or universities. 

b. Make grants to State Boards. 

c*. Make grants to other public or non-profit private 

agencies and institutions. 

d. Make contracts with public or private agencies 

organizations or institutions. 



To: A. 

B. 

C. 

D. 

E. 

F. 



Develop and disseminate vocational education 
curriculum materials. 

Develop standards for curriculum development^ 
Coordinate efforts of states and prepare lists of 
available material. 

Survey curriculum development in other agencies. 
Evaluate curriculum materials and their use. 
Train personnel in curriculum development. 



Furthermore, the Act defines curriculum materials as follows: 

" .curriculum materials means materials consisting of 

a' series of courses to cover instruction in any occupational 
field in vocational education which are designed to prepare 
persons for employment at the entry level or to upgra e 
occupational competencies of those previously or presently 
employed in any occupational field. 



Our task at this national conference is to study the sections of the 
Act related to curriculum development in vocational and technical education. 
We must develop guidelines for interpretation and action concerning each 
sectiorfof Part I of the Act. The success of the conference depends largely 
upon the productive work of small groups. Experts have been asked to 
provide special insight into elements directly and indirectly rela e o 
particular facets of curriculum development. The considered judgment 
of this conference is an imperative necessity. We must be in a positio 
to forth our best ideas; in particular we must Provide a succinct 
description of the role to be played by the Division of Vocational and 
Technical Education, Office of Education. 
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Following this conference nine regional clinics will be held. 

Each clinic will inculde a review of the nine national conferences. Each 
regional clinic will have suggestions to make about curriculum development. 
When all of the meetings have been concluded the staff of this nationa 
conference will summarize the total knowledge gained from this national 
conference and the regional clinics and will prepare guidelines for 
curriculum development in vocational and technical education, which 
will be widely distributed. 

Our task is to answer major questions such as who? what? why? 
where? when? and how?, as these questions are related to the sections 
of the Act Our judgments will provide a frame of reference for the 
action that the Commissioner of Education must take in the near future 
and will influence the course, direction, and emphasis upon curriculum 
development for a long time in the future. 



References: 
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Education. Education' for a Changing World of Work, OE-80021. 
Washington: Government Printing Office, 1963, pp. 240-241. 

2. U.S. Department of Health, Education, and Welfare, Office of 
Education. Vocational Education: The Bri dge Between Man and^ 
His Work, OE 80052. Washington: Government Printing Office, 

1968, p. 209. 
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March 5, 1969 



THE FUTURE OF VOCATIONAL CURRICULUM DEVELOPMENT 

by 

Gerald B. Leighbody 
Professor of Education 
State University of New York at Buffalo 



The Vocational Amendments Act of 1968 provides, for the first time, 
special funds for curriculum development. These funds may be used for^ 
several purposes, all related to the improvement of instruction, and with 
special emphasis upon new and emerging occupations. 

It would appear that the present need is to assess the nature and 
scope of the curriculum task that confronts the vocational educator as 
a result of this legislation, so that suitable plans of action can be 
prepared and the necessary organizational arrangements developed. But 
we will not be ready for detailed planning until certain fundamental 
issues are clarified and certain basic concepts re-examined. Therefore, 
this paper will not attempt to suggest the organizational strategies or 
working procedures which will ultimately be necessary to get the job done. 
Instead it will focus on some very basic considerations, and will raise 
some critical questions which must be answered before useful goals can 

be set. 

We will begin by taking a look at where we are now. We will then 
turn to the need to define and delimit the task ahead, for it would be 
as easy to overestimate as to underestimate its dimensions, and resources 
are not unlimited. Finally, we will discuss some general, but very 
relevant issues which must concern the modern vocational curriculum planner. 



Where We Are Now 



Curriculum development in vocational education is now perceived^ 
chiefly as the process of preparing and producing instructional materials. 
These materials usually take the form of a syllabus or similar type of 
content document outlining what to teach in a particular occupation. 
Sometimes suggestions for equipment and facilities required to teach the 
course are also included. Many such materials are prepared by individual 
teachers or groups of teachers on a local basis for local use. There is 
no way to know how much instructional material of this kind exists because 
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t is not usually available for distribution. Some states, and some 
universities have curriculum centers -which develop and produce limited 
kinds of materials. The United States Oifice of Education has produced 
a certain number of publications relating to curriculum, but its funds 
have been inadequate for any important contribution. The major sources 
of instructional materials for vocational education are the commercial 
publishers and the manufacturers of specialized teaching devices which 
are popular with many teachers for teaching certain occupations. For 
instructional materials vocational educators must continue to rely, 
chiefly on the commercial producers. Certainly there is no justifica- 
tion for establishing, with public funds, any competing enterprises for 
developing and producing instructional materials, and the wording of. the 
Vocational Amendments Act does not seem to contemplate such an activity. 



In any case, the most serious need is not the production of teach- 
ing materials. The real problem is the continuing tendency to confuse 
instructional materials development with curriculum development. 



Curriculum is the sum total of the learning experiences for which 
the school has responsibility, whether they occur in school or not. 

To plan a curriculum means to select, arrange and sequence these experiences, 
through the joint decisions of teachers and learners, so that successful 
learning results. In vocational education, as in other areas, this 
requires that learning outcomes be clearly defined, in behavioral terms, 
and suitable evaluative devices designed to measure their achievement. 
Vocational educators will find this increasingly difficult to do, because 
/ocatioual success in our society is so dependent upon general educational 
development that the skills of work often cannot be identified from other 
life skills. The laundry list of job operations and related technical 
information that was once considered the standard content for vocational 
courses is now quite inadequate. If we direct our efforts at this level 
in attacking our curriculum needs, we shall be v ^sting our time and our 
new resources. 

Vocational curriculum planners, like others, must start with basic 
educational decisions which lead to sound educational policies. These 
decisions must take account of at least four major determinants. These 
are the nature and needs of our society, the nature and needs of the learn- 
er, the - nature of the learning process, and the nature and role of the 
teacher. Vocational curriculum makers should apply the best knowledge 
available about all of these in order to help people achieve useful, 
satisfying work lives. Until this kind of planning takes place there is 
no need to concern ourselves with materials or methods, to evaluate exist- 
ing materials or recommend new ones. 



The Size of the Task 

Before we can estimate the magnitude of the curriculum task we need 
to re-examine carefully some current assumptions about vocational education 






It i!> often defined as that education or training that leads directly to 
a job. This is taken to mean that nothing that does not prepare an 
individual for a given job can be considered as vocational education. In 
fact this statement is made in an editorial in the January 1969 issue of 
the American Vocation Journal. This same editorial page states that voca- 
tional education now has the responsibility for occupational education for 
all persons who do not attain a baccalaureate degree. Nov/ if we accept 
both of these assumptions, exactly what would this mean in practice? 

To begin with, how are we to know ahead of time who will terminate 
his education at what point -- who will complete a baccalaureate degree 
and who will not? And if we could know, are we to suppose that every 
such person is at some point to be enrolled in vocational courses, in a 
vocational school or program? How many thousands (not hundreds) of 
occupations would have to be included in vocational programs if this was 
to be attempted? How justify the fantastic costs of duplicating, in 
school settings, the environment, facilities, and equipment to train for 
these thousands of different jobs, to say nothing of the huge number of 
teacher specializations? Is it not true that the great majority of jobs 
in our economy can never be taught in school, but can be learned only on 
the job? As we try to answer these questions we begin to" see that we must 
modify some of the grandiose and extravagant statements about the task, of 
vocational education which are now finding their way into print. If it 
is true, for example, that twenty per cent of our young people complete a 
baccalaureate degree, we are not entitled to conclude that all the rest, 
or any major part of them, should be enrolled in a vocational course. Yet 
this argument has been repeatedly advanced in recent statements. It 
reflects simplistic and uncritical thinking. 

In the vocational technical centers and area schools being establish- 
ed at the present time the course offerings are typically limited in^ 
number, traditional and stereotyped. A center which offers preparation for 
as many as twenty occupations is unusual and among these the long establish- 
ed trades and mechanical occupations predominate. So long as a small 
percentage of students enroll in vocational courses, and, fewer still complete 
them, the discrepency between the distribution of jobs in the real world and 
the jobs represented in vocational programs is not apparent. However, if 
the numbers entering and completing such courses should increase greatly,^ 
the gap would be glaringly evident. Vocational educators should think twice 
before they advocate vocational education in its present form for all youth 
who do not go to college. However, they are not likely to be called upon 
to undertake such a program. 

We shall have to determine clearly what job training can best be done 
by industry, business and government employers, and what can best, be done 
in schools. It will be found that most jobs can be learned only on the job, 
not in schools. To serve all who need preparation for work, vocational 
educators will need to form educational partnerships with business, industry 
and government so that most of their students can receive their vocational 
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continuum here that does not permit separation into discrete stages. 
Vocational educators cannot continue to assume that guidance counselors 
are responsible for career orientation, and then stand aside and wait 
for properly oriented students to arrive on their doorstep, ready to be 

trained. 

The federal vocational education acts of 1946, 1963, and 1968 have 
all provided funds and authority for vocational educators to enter actively 
into the area of career orientation. Yet it has never been done. . Now 
vocational curriculum makers must make it a matter of first priority. A 
policy which assigns to vocational education a supplementary or advisory 
role rather than a major responsibility for career orientation is a mis- 
taken policy. There is no area of education that has a greater stake in 
the process of career planning and orientation. To suggest, as has been 
done recently that this is a secondary concern of vocational educators, and 
should not make undue demands upon their resources, is irresponsible. In 
discussing it. Congressman Pucinski said recently, 

’'Vocational education has substituted sheer job-skill 
training for genuine career development. There is a 
vast difference between these two orientations. 



We would be justified in directing most of our vocational curriculum 
efforts below grade twelve toward the area of career education and planning 
rather than preparation for particular occupations. This is strongly 
suggested by the findings of Project Talent. This study shows that in 
following up the career plans and choices of 400,000 boys who expressed 
their career plans in grade eleven, seventy-five percent of them had changed 
those plans substantially four years later. 



The Involvement of Teachers 

Curriculum cannot be separated from instruction, and therefore from 
teachers. It has long been customary for many vocational teachers to be 
active in curriculum development, often because they were the only real 
experts in their subjects. More recently, many other teachers have taken a 
renewed interest in curriculum decision making, and their active partici- 
pation is now frequently sought when teacher organizations negotiate their 
privileges and benefits with school authorities. Vocational curriculum 
development must involve teachers fully at every stage. 

Curriculum decisions made by so-called experts and passed along to 
teachers have seldom found their way into classroom action. The greatest 
dust collectors on the classroom shelves have been the well intended 
course outlines, teachers guides, and similar materials prepared by 
curriculum specialists. The only curriculum a teacher is likely to take 
seriously is one he has helped to plan. The more competent and professional 
the teacher, the more this will be true. Vocational funds spent to 
improve instruction through curriculum development will have little effect 
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unless the teacher has a leading part in the process. This means that 
curriculum improvement must be closely coordinated with teacher education 
and calls for the greater professionalization of vocational teachers. 
Vocational teacher education has often placed too much emphasis on 
occupational skills and too little on other professional competencies in 
selecting and training teachers. In the future, if vocational teachers 
are to contribute as they should to curriculum improvement, they must 
possess a better balance of professional qualities. 



Summary 

The task ahead for vocational curriculum improvement is a large 
one, but its dimensions are not so much in numbers as in quality and nev; 
directions. To meet its responsibilities, vocational education will need 
to do more than just serve more people. It will have to re- think its 
purposes and realize that it can achieve its time honored goals only by 
making them relevant to today's world and the fast approaching world of 
tomorrow. This will involve not just more, but different and better 
vocational education. To put it plainly, the demand is for real reforms 
and sharp departures from the past, not so much in goals as in means 
for reaching them. We have nev; resources with which to work. We had 
better use them wisely. 
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Curriculum Development in Vocational and Technical Education 
EVALUATION OF CURRICULUM MATERIALS AND THEIR USE 



W. James Popham 

Center for the Study of Evaluation 
University of California, Los Angel-es 



What the classroom teacher has always known is now 
being perceived by an increasing number of influential 
American educators, namely, that the single most impor- 
tant determiner of the curriculum is the set of curri- 
culum materials on which instruction is based. Of course 
there are some teachers whose instructional efforts are 
carried on almost apart from curriculum materials. But 
for most teachers the text book or its equivalent is 
the prime mover with respect to what happens instruc- 
tional^. This state of affairs should surprise no one. 
Teachers are usually human. And most human beings adopt 
the route of least resistance in coping with any kind 
of problem. The easiest course of action for a teacher 
to take in discharging his instructional responsibilities 
is to adopt the ready-made structure provided by cur- 
ricular materials. 

These observations regarding the influence of cur- 
riculum materials have not been made to denigrate the 
practices of teachers, only to accentuate the importance 
of evaluating such materials properly. Since the in- 
fluence of curriculum materials is enormous, any evalu- 
ation of them must be commensurate with their potential 
impact. 

Common Criteria 



How are curriculum materials currently evaluated? 

It would be inappropriate to attach the label "sys tematic" 
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to the manner in which curriculum materials are judged 
these days. While unsystematic, however, it is possible 
to isolate certain criteria which are employed by materials 
evaluators . 

The Content Criterion . When an educator considers 
the possible adoption of a new curriculum artifact, for 
example, a textbook, filmstrip, or recording, a chief 
factor used in making that decision is the adequacy of 
the content. Is the content of the textbook, for instance 
consonant with the best current thinking of experts on 
the topic? Suppose that a new text in electronics is 
available which advocates an approach to troubleshooting 
that is passe.' Obviously the potential user will be 
reluctant to adopt an outmoded point of view. But that 
is a fairly obvious situation. There are invariably 
instances when the potential user’s own preferences re- 
garding what content emphases are most crucial will in- 
influence the evaluation of curriculum materials. ’’Did 
the illustrated filmstrip on the principles of carburetion 
emphasize the things I believe important?” This is the 
way a teacher might phrase such a question. For years, 
the content criterion has probably been the most in- 
fluential factor in evaluating curriculum materials. 

The Cosmetics Crite rion. Another factor which 
affects the evaluation of curriculum materials is the 
manner in which the materials are packaged. Are they 
attractively put together? Cleverly illustrated? Are 
the figures and graphs easily understood? Such questions 
figure prominently in the evaluation of curriculum materials 
made by many educators. 

The Charisma Criterion . Although charisma is 
usually associated with the political arena these days, 
there are authority figures in all fields whose prestige 
alone can positively influence the evaluation of cur- 
riculum materials. Merely because a text was written 
by a super- stature specialist in the field is sufficient 
for some evaluators. 

Thus we find prominent among current criteria those 
of content, cosmetics, and charisma. It is distressing 
to realize that considered separately or as a pooled 
collection these criteria are completely inadequate for 
the proper evaluation of curriculum materials. 
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A Defensible Criterion 

The most defensibl e criterion by which to judge t he 
adequacy "of curr iculum materials is the degree to which 
t hose materials, l'f used as directed, can consistently 
Bring about "c Tesi fed changes in the behavi or of the in- 
tended learners. If one grants the assumption that the 
principal" purpose of instruction is to promote desirable 
modifications in the behaviors of learners, then it seems 
only reasonable that to be considered valuable, cur~ 
riculum materials must be of assistance in that endeavor. 
To use such a criterion with any rigor, we must be con- 
cerned with 'easurable, not ephemeral, modifications in 
learner behavior. We must know precisely how we want the 
learner to behave after interacting with the curriculum 
materials, then see if he can behave that way. 

Schematically, we can represent this approach to 
the evaluation of curriculum materials as follows. 



1 Designate 


N. 


Assess 


s. 


Use Curriculum 


V 

/ 


Assess 


1 Objectives 
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Learners 
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Materials 




Learners 



Figure 1. A Scheme for Evaluating Curriculum 
Materials 



The first step is either to construct or select a 
set of operationally stated instructional objectives 
which it seems reasonable to expect the curriculum 
materials to accomplish. Measures of pupil performance 
based on these objectives must also be prepared or selected 
The second step is to assess the degree to which the 
learners can already perform the behaviors delineated m 
the intended objectives. This pretest is crucial to es- 
tablish clearly that prior the interaction with the cur- 
riculum materials the learners could not already display 
the intended behavior changes. The next step is to allow 
the learners to use the curriculum materials as_ directe d 
by the developer of those materials. The matter. of usage 
is becoming increasingly important, for if materials are 
to be used with immense variablility , we should not be 
surprised if the results of their use are also tremendously 
variable. Those concerned with the development of cur- 
riculum materials are now being urged to specify the. essen- 
tial ingredients of the procedures which they hope will 
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be employed by users of the materials. The final step 
involves^post testing learners to sec whether the objec 
tives have been achieved. 

Returning to the previously discarded criteria we 
can see that considerations associated with the content, 
cosmetics, and charisma are encompassed by a focus on 
the outcomes of instruction. To illustrate, we may want 
a learner to display a behavior which is consistent with 
the best that isVown in a field at the moment 
test's content is inappropriate it seems unlikely that 
^he hoped fo? behavior will emerge. Similarly cosmetic 
considerations may or may not be relevant to the lea 
attainment of a given objective. In some cases P^tty^ 
pictures may distract rather than abet. And 
?hat even the most prestigious of authorities can turn 
out an inadequate textbook. All three of these con 

siderations, then, are inappropriate because they aie 

related to the means, not the ends, of instruction. A 
proper evaluation of curriculum materials must focus on 
whether materials produce worthwhile behavior changes in 

the learner. 

Implementation 

The specifics of how such an approach to evaluation 
would be carried out in the schools is beyond the scope 
of this paper. The best treatment of the topic is^ti 
the reoort* prepared a few years ago by a joint AbKA 
APA-DAVI Committee dealing with the utilization of P^o 
grammed instructional materials. Just a few approaches 
will be mentioned here. 

Evaluation Based on Publisher Data . Ideally, the 
publishers of curriculum materials would supply poten ia 
Ssers with (1) sets of behaviorally stated objectives 



* American Educational Research Association, America 
Psychological Association, Department, of Audiovisual In- 
struction Joint Committee on Programmed Instruction and 
Teaching Machines , February, 1963, Audiovisual Instruction. 
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their materials were designed to accomplish (2) specific 
directions for use of the materials, and (3) validation 
data based on tryouts of the materials with learners 
whose characteristics were clearly explicated. Then, 
the potential user could judge the similarity between 
his learners and those in the publisher's field tests 
and decide whether the probable success was sufficient 
to warrant his acquisition of the materials. But much 
economic pressure will have to be brought to bear on 
commercial publishers before they routinely produce 
such data. 

Evaluations Deve l oped by Neutral Agencies . One of 
the more interesting~cTevelopments concerning the evalua- 
tion of curriculum materials has been the establishment 
of the Educational Products Information Exchange (EPIE) 
Institute, an independent, nonprofit organization char- 
tered in 1967 to serve educational decision-makers in 
schools and industry by s-upplying information about the 
availability and effectiveness of instructional materials 
and equipment. While the publications of EPIE since 
its establishment (under an original grant from the U. S. 
Office of Education) have .focused largely on the evalua- 
tion of educational equipment (particularly of an audio 
visual nature), the institute plans to undertake system- 
atic evaluations of educational curriculum materials as 
well.' Thus far the majority of their publications re- 
garding curriculum materials have unfortunately been 
descriptive rather than evaluative.. With the October, 

1968 issue the institute’s journal (formerly called the 9 
EPIE Forum) is now known as the Educationa l Product Report . “ 
It is reasonable to expect that If EPIE and similar 
organizations emerge, they w ill provide educators with 
something akin to a Consumer’s Report for curriculum 
materials and related products. Such reports, if systematic, 
sensitive, and objective, should clearly be of consider- 
able utility in choosing among competing curriculum 
materials . 
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The Educational Product Report is published nine 
times a year by the Educational Products Information 
Exchange Institute, 386 Park Avenue, South, New York, 
N. Y. 10016. 



The generally beneficial impact that programmed 
instruction has had on the quality of student learning 
has provided measurement and evaluation personnel wit 
a number of problems regarding the manner in which the 
attainment of educational outcomes should be aosessed. 

Of particular interest are those investigations con- 
cerned with the difference between norm-referenced and 
criterion-referenced measurement approaches. Inese 
approaches are being employed by material developers 
themselves . 

Evaluation by the User. For the time being the ^ 
user wiTTpi^FaFry^The one called on to do most of 
the evaluation of curriculum materials. This will re 
quire the identification of specific objectives, suit- 
able measures, and a thorough committment t0 a * 
oriented, i.e., criterion-referenced abroach i to eval 
ation. A school or school district would have to f^rst 
purchase a modest number of the curriculum materials, 
then conduct its own field trials, prior to making a 
maior purchase. But to do this systematically in all 
districts is a Herculean effort. We must maice it easier 
for educators to engage in criterion-referenced instruc- 
tion and evaluation. 

. 3 

Criterion- Re ferenced Ins t rue t ion 

As indicated earlier, criterion- ref erenced instruc- 
tion Reuses primarily on’the degree to which the learner 
can perform specified criterion behaviors. For exampi , 

d n ? drwhit g to n revise i orkrb!sit 1 of t lLrnIr performance 
data f not according to the judgment .of^co^sulting^xperts 

select^ne^se^o^supplementary 0 reading texts instead 
of another because of pupil performance on re a e cr 
terion tests, not because one set of texts is mo 



3 The remaining remarks are based on a symposium 
presentation at the Annual American Educational Research 
Association Meeting, Los Angeles, February 5-8, 1969. 
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attractively illustrated than the other. Such examples 
accurately suggest that a primary feature of criterion- 
referenced instruction is a preoccupation with the r 
suits of instruction, not the procedures used to promote 
them It reflects an ends - oriented approach to mstruc 
tion* rather than means-oriented approach. Since most 
educators concur that the ultimate index of an educa- 
tional program's worth is the degree to which it bene- 
fits the learner, the increased support of criterion- 
referenced instructional approaches is gratifying. 



But against the increasingly supportive backdrop, 
it is distressing that very few large-scale criterion- 
referenced instructional operations are underway. Verbal 
support is there. Widespread practical implementation 

there is not. Why? 

A Time-Consumi ng Task . The principal ceterrent to 
expanding the extent of criterion- referenced approaches 
used in the nation's schools is fairly easy to identify. 
Developing criterion measures of- sufficient quality and 
satisfactory breadth is too much work for most educators. 
Developments regarding the use of behaviorally state 
educational objectives may be instructive here. 

Much of the recent agitation regarding the des lia- 
bility of describing instructional objectives in terms 
of measurable learner behavior is based on the belie 
that operationally stated objectives will more readily 
permit educators to assess the impact of instruction 
where it should be assessed, namely, in modified learne 
behavior. But many proponents of operationally state 
educational objectives are beginning to complain abo 
the paucity of such objectives in the schools. Educators 
can be informed of the merits of behaviorally stated 
obi ecti^es ; they can be taught to state objectives P r J 
perly ; they can even become quite enthusiastic about the 
desirability of stating objectives behaviorally. Bu- 
few of them do it. The reason is not unwillingness but, 
instead, reflects a lack of wherewithall . Teachers are 
already too burdened to find the time to develop opera- 
tionally stated objectives for their classes. School 
districts have already committed their increasing y ^ 
limited resources to other tasks. In those isolate in 
stances where there has been an effort to develop P ® 
instructional objectives on a large scale, the participa- 
ting educators will readily admit how taxing the enter- 
prise has been. 



Imminent Duplication. The financial and personnel 
costs"oTTHe isolatecFprbjects to develop instructional 
objectives points up another problem. In spite of the 
difficulties associated with the development of explicit 
objectives, some districts are undertaking the task. 

For example, several months ago the Clark County, Nevada 
School District developed a set of behaviorally stated 
objectives for mathematics instruction, grades K through 
6. There are other examples of such endeavors in various 

parts of the U. S. 

The absence of any scheme through which one district 
could become aware of the existence of similar develop- 
mental projects makes it probable that a distressing 
amount of duplication will occur among those few educa- 
tors who are zealous enough to attempt the development 
of precise instructional aims. For instance, moie than 
a year after the Clark County , Nevada schools hod com 
pleted their preparation of K-6 instructional objectives 
for mathematics, two districts in different states com 
menced work on precisely the same project. They were 
unaware of the Clark County objectives. The wheel was 
about to be re-invented. 

Not that the Clark County objectives would satisfy 
all districts; undoubtedly there would be modifications. 
But the energy that could be saved nationally by adapting 
extant sets of objectives rather than starting from 
scratch, is incalculable. For example, several of the 
USOE-supported regional laboratories are investing sig- 
nificant resources in encouraging educators to develop 
operationally stated goals. The probable overlap be- 
tween such efforts and similar projects initiated y 
local districts is considerable. 



Objective-Generators and Objective-Selectors 

It has become increasingly clear to those who have 
been promoting the use of operationally stated objectives 
that it may be expecting too much to ask already harassed 
teachers and administrators to generate their own objec- 
tives. It is an arduous task 'and, although the teacher 
may be willing to state his objectives behaviorally, unde 
present conditions most teachers just can t find the time 
to do it. But though objective -generation may be too 



21 



demanding , objecti ve -select ion should not be. If the 
instructor’s tasTT were simply to choose from compre- 
hensive sets of operationally stated objectives those 
which he wished to achieve, his task would be manageable 
He could follow through on his commitments to precisely 
explicated goals without being obliged to construct all 
such goals himself. But, obviously, someone needs to 
construct the objectives from which h^ can select. 



Local Option. Under any scheme in which the educa- 
tor is tKe~se 1 e c t o r rather than generator of objectives 
there may be some concern regarding the degree to which 
the objectives will be "imposed from above. A viable 
objectives selection scheme, however, should permit just 
that-- the selection of objectives. If particular ob 
jectives are not preferred, they are not selected. ** 
all of the objectives are not available which the selector 
favors, he can always generate additions. Having selec- 
ted the bulk of his goals from those prepared by others, 
such an objectives generation task should be manageable. 
Local autonomy in the selection of objectives should be 
an integral part of any objectives selection scheme. 

The availability of objectives from which to choose 
should increase the educator’s range of alternatives, 
never decrease his self-direction. 



Objectives Plus Criterion Measures 



Another factor which has not been perceived by all 
advocates of precise objectives is that they may be 
necessary, but by themselves they are far from sufficient. 
Too often even a behaviorally stated objective may ( be 
used as window dressing for "instruction as usual. A 
precise objective can be most helpful wnen planning an 
instructional sequence, since there is ciarity regarding 
the intended post- instruction competencies of the learner, 
But an explicit objective becomes even more useful when 
we evaluate an instructional sequence. This can be accom- 
plished bv ascertaining the degree to which the objec 
tive has 'een achieved. To perform the latter function 
we need measuring devices based explicitly ^ on the objec- 
tive. A criterion-referenced approach to instruction 
requires criterion measures . 




